DOCDHEIT BBSOHB 



ID iai 420 

AUTBOB 
TITLE 

IHSTITOTIOH 

SFONS AGENCY 
lEPORMO 
PDB DAT] 
BOTE 

EDBS FBICE 
DBSCRIPTOHS 




IDENTIFIERS 



TB 006 418 

Alley, Hilliaij E. ; Gibson, Terry A. 

Predicting Success- in the APROTG Scholarship Program, 
•Final Report, January 1976-January f977« 
Air Force Human Resources Xab., Lackland AFB, Tex. 
Personnel Besearcb Div. 

Air Force Human Resources Lab., Brooks- AFB, Tex^s. 
AFHRL-TB-77-11 
Apr 77 

aop. _ ' . . 

MF-tfO.83 HC^$2.06 Pliis Postage. 

Ability Identification; Aptitude Testp; College' 
Majors; Colleges; College Students; *CoBpetitive. 
Selection; Comp^^ity Level; ♦Dropout Prevention; _ 
Flight Trainingi^igher Education ; Hilitar.y 
Personnel; Military, Training; Predictive Validity; 
♦Predictor Variables; ^Scholarships; Student Ability; 
♦Success Factors; lests; Universities 
Air. Force Officer Qualifying Test; ♦Air Force Reserve 
Officers Training Corps ^ ^ 



lESTRlCT ' • . 

' Research was conducted to improve applicant selection 

procedures in the Air Force Reserve Officers^ Training Corps (AFROTC) 
scholarship program.. The objectives were to document predictive 
re^atioiiship£i, if any, between student aptitude measures, academic 
major,- institutional selectivity, and eventual .completion of the 
scholarship program. The analysis included recbrds ot student 
performance foi: all AFfiOTC participants duriligf FT 71 throi^gh FY 75 (N 
= 23,000). Results indicated that success in the program could be 
forecast with prior knowledge of: (a) applicant scores on the Air 
Fbrce Officer Qualifying Test {AFOQT)*OQ composite and (b) the 
intended, academic major (science and engineering versu^^ other) . A 
significant degree of predictive accuracy was achieved for each of 
'the 4-year, 3-year, and 2-year programs. The effects of additional 
information based on the AFOQT (Pilot and Navigator fJomposites) and 
institutional selectivity was f^und to be non-essential for 
predicting ECIC training outcomes although the use^tilness of the 
rdted training composites, was again doct^mented for/undergraduate 
pilot and navigator training. Specif ications were discussed to 
developing a dual selection system appropriate for both pre- and 
post-entry training programs. The implementation' of such a system 
could result in considerable savings given the high costs of 
attrition typically associated with these programs. (Author) 
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PREDICTING SUCCESS IN THE AFROTC SGHOLARSfil^ PROGHTAM 




I. INTRODUCTION 

-A tignificant proportioji^f expenditures in the Air Force Reserve Officer Training Corps (AFROTC) 
are devoted to> the CoU^^i^cholarship Program, wliich each year underwrites 6,500 scholarships at an 
annual cost of (approximately 14 million dollars. The mayiitude of that investment requires efficient. 
r,,^managernent to-avoid unnecessary losses due to attrition, to maintain the quality of graduates, and, in 
ijpneral, to achieve an optimum' return for each dollar invested. 

The ptograpn authori^s scholarships of varying lengths: 4-year, 3-year, and 2-year, each representing 
the total costs of tuition, fees, and books^'pr the designated enrollment pcriod.'Scholarship recipients atsp 
receive a monthly stipend of $100. While the annual expenditures pei litudent vary by school, they 
typically range between Sl,200 and $3,000 with an average value of apdroxiniately $2,000. Historical'HaU 
on student attrition^NUi the 4*year program shows that approximately half of the students awarded 
scholarships failed to complete the program for various academic and other reasons. The attrition rates/or 
the. 3- and 2*year programs are generally less (averaging 15% and 12%, respectively) ^s a result of the higher . 
loss rates associated with the first academic year. 

' In 1975, Headquarters AFROTC commissioned a study of the scholarship award procedures with a 
view toward reducing student attrition in the program. This could be accomplished if a certain proportion 
of **higt} risk*^ applicants (i.e., those with little probability of completing training) could be identified prior 
to actually awarding scholarship benefits. The research was to be conducted in two phases. DuringPh^ise I, 
the basic feasibility of establishing selection criteria was to be examined using historical training records for 
a 5-year period. The principal objectives ^Ve re: 

1. To document etnpirical relationships, if any, between individual student aptitudes and probability 
of successful completion, ? \ 

2. To examine modifying influences attributable to the type of academic m^or and to the overall 
difficulty level of the school, aihd 

3. /To determine the pdteiptial applicability of the procedure to the 4-, 3-, ai>d 2-year scholarships 
programs. 

During the second phase, selection procedures would be refined using an expanded predictor set which had 
not yet ipatured. The purpose of this report is to document the interim fmdings from Phase I. 

, H. APPROACH 

Subjects • ^ 

Records of final training disposition for AFROTC participants during FY 71 thro.ugh FY 75 served as 
the basis for analysis (N = 22,663). These people were enrolled during that time *period at 175 U.S. colleges 
and universities offering AFROTC programs^ Moreover, they had either successfully conripteted the program 
or had disenroiled for academic or motivational reasons. The schools are considered representative of all 
mqpr academic institutions in both the pubhc and private sectors. A distribi^tion of students by program 
category indicated 10% were 4-year scholarship rccipienti^^5!p= 2,235), 11% were 3-year scholarship 
recipients (N- 2,482), and 79% were either 2-year scholarship, recipients or nipn-schdarship contract 
stu<fents enroUed in the advanced AFRpTCi^rpgrarnXN =J7^94$),JlicJatter- two categories^ were 
in the same sample si^Tce both w^re required to sign J-yeaj^ppntractscoritingentoD successful completion of 
Uiejunior and senior years of acadenu^ study. ^ , , ^ 



Pitdlctoivtriiblw 

Variables for the analysis, imlcss otlicrwisc indicated, were obtained from historical records of 
AFROTC participants nuintaiiicd at the Air Force Human Resources I-aboratory. 

L Aptitude Measures: Individual aptitude measures for each participant were obtained from the Air 
Force Officer Qualifying Test (Al'OQT).*a group-administered battery designed to evaluate aptitudes which 
arc important for conuuissioned officer performance and success (Miller, 1968. l%9). The AFOQTconsisU 
of 13 subtests, each included in one of several test booklets. Ttie subtests taken in various combinations 
yield titree composite measures in percentile form as shown in Table I . 



Table I. Subtests and Composites 


of the AFOQT 






APlHud* CompOflHl 




Offldcr 






SuMMt Quality 


niot Nav'T«»h 


Quantitative Aptitude ' X 




X 


Verbal Aptitude X 






Officer Biographical Inventory X 






Scale Reading 




X 


Aerial Landmarks 




X 


General Science ^ 




X 


Mechanical Information 


■ X 


X 


MechanicaJ Principles 


X ♦ 


X 


niot Biographical Inventory 


• X 




Aviation Infbrmation 


X 


• 


Visualizationlof Nlaneuvere 






Instrument Comprehension 


X 




Right Orientation 


X 





a. Tfie Officer Quality (OQ) (b mposite -The OQ composite is primarly a measure of general 
learning abUity and officer quality. .It contains measures pf verbal and quantitative aptitude, reasdning 
ability, background knowledge relative to world events, and an inventory of biographical material predictive 
Qf officer leadership. AppUcants with high Officer Quality scores rtTay be expected to do weU in any 
technical training program having appreciable academic content. 

b. The Pilot Cbmpo^-Thh is a measure of some of the characteristics necessary for successful 
completion of pilot training. It includes subtests of mechanical experience, spatial information, and abUity 
to understand and interpYc;t information received from aircraft instniments. Applicants with high scores on 
this composite have considerably better chances of completing pU^t training than those with low scores. 

c. The Navigator-Technical (Nav-Tech) Cbmposite-Tht Nav-Tech comp9site is a measure of 
abUities to interpret dials ai?d tables, to understand scientific and mathematical principles, and to 
comprehend mechanical and jpatial concepts. It is designed to predict success in trainiitg courses requiring 
these abUities such as navigator traininjg. communications, electronics, maintenance, engineering, and 
technical intelligence. ■ " - 

2. Institutional 3fjectivit^: This variable was designed to take into account the varying degrees of 
difficulty presumed 4<^exist^et^n^ acadeHttc4nstittttioB&^h^^ detaehmente. ft w» defined 

operationally on the basis of the average American CoUege Test (ACT) scores for entering freshmen 
(American Council- on Education, 1968) at each host college or university.' In the analysis, all persons 
enrolled at the same academic institution received identical selectivity scores / * 




3. Academic Mi^Qt: As i generti; index of the effect of iCidemic tpecitlty on probability for Ween, 
*'$0 cadet academic majors were grouped into two mutually excluaive categories-Science and l^giikering 
(S&E) venua Non-Science and Hngineering-aa shown in Tibk 2. These definitions are consistept with tl)e 
AFROTC practice of identifying certain specialties which^i^ of particular interest in subsequent active 
duty attlinments. These academic specialties are also known^al Category III minors. 

, nbit 2. Academic Specialties Designated Science and Engineering 
A— ^mtoM«i»r apttUHitt 

^ Science and Engineering Aeronautical Technology, Aeronautical Bngineering, Aerospace' Engineer- 

ing, Aitronautical Engineering, Civil Engineering, General Engineering, 
Industrial En^neering, Mechanical Engineering. Architectural Engineering. 
Architecture^ Electrical Engineering, Electrical Technology, Communica- 
} tions Technology, Computer Sciences, Mathematics,'Physics, Space Physics, 

Meteorology 

Non«Science and Engineering All other academic specialties 



Criterion Variable 

Training outcome defined on the basis of graduation versus elimination served as the principal 
ariterion^r developing the selection system. The elimination category included sJI types of disenrollmcnt 
for any reason including academic, motivational < physical, etc. 

Afialyiea 

Individual student, records were partitioned into three samples defined on the basis of program 
length: (a) 4'year schiolarship recipients, (b) 3-year scholarship recipients, and (c) 2-year scholarship 
recipients and other .qon tract studenis. Withirv each sample, separate regression analyses were conducted to 
determine the effects of the prec^or variables on training outcome (graduated vs. eliminated). The 
functional relationships initially deflne^nay be expressed as follows: 

Training Outc6me = f (Aptitude x Institumnal Selectivity x Academic Major). 

In defining the initial prediction model, a large number dfinonjj^ear and interaction terms were 
generated from tlie primary variables to insure relatively complete investigation all possible relationships. 
'Po test for efTects attributable to specific predictor measures, several reduced models were also.constructed 
in such a way diat various components of the initial prediction model could be systematically eliminated 
from consider^ipn. Comparisons, baset) on the statistical accuracy of each model, were performed using 
the F-ratio and assodated probability values (Ward 'A. Jenir^g^, 1973). Complete specifications for th^ 
analysis including variable descriptions, predicti<m modeb, and specific comparisons performed ait given in 
Appendix A. . - . 

• ' , 

DL MOULTS AND dIiSCUSSI^ 

Basic descriptive data by subsample (Table's) indicated that people enrclled in th^ various programs 
^difTeied in a numt)er of respects. The average OQ score in tl^e 4-year program was 61 versus 68 in the 3-year 
pmgiaju and 5 7 in the 2-ye ar/other program, ^res on the Pilot composite were somewhat more consistent 
acroM groMpt averag^lng 57, 58, and 55, respectively. Thp higKest average Navigator-Technical score was 
obierved in the 3-year schoUrship group (64) with lower averages being noted in the 4-year and 2-year . 
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Tahh' J. Mci^nib iind Slindurd I>cviarioa<i of Prinmry Variable* 



ScholinMp Profram 



Four V««f Thr«« YMr Two VMr/bth«r 

(N * (N * a,4aa) (N-i7,t4a) 







so 




so 




so 


AIOUT OO 




; 2.VV^ 


()K.42 


22.18 




26.^3 


AFOUT hlot 






- 5K.25 


24.82 


55.47 


27.13 


AFOgT Nav/Tcch 


5H.45 


22.4f> 




22.^5 


'51.04 . 


27.^)8 


SAh v$ Non-SAI 




.4(> 


.31 


Ab 


.18 


.38 


Average ACT 


2.VK2 


2.2(> 


2.V.^4 


2.4« 


2.V05 


2.52 


AFROTC Completion 


50 


M) 


• .H5 


.36 


' .88 


.33 



cohorts (58 vs. 52/ Tlic proportion ol sticrac ;nul engineering students in each program was e(|uiva!ent for 
both the 4. and 3-ycar groups api)rOMttutcly 30':. Only 18% of tlie 2-year/other scholarship students 
were designated science and cnginecrinji majoril. Avera^;c ACT was essentially identical for all groups. The 
overall completion rate for students awarded 4 yc4r scholarshi}^ was 50%. Completion rates in the 3- and 
2-year programs were 85'" and 88'?. Tlicsc data highlight the higli rate of attrition normally associated with 
the freshman academic year. 

Results of the regression analysis to determine the unique cffect^ssociatcd with aptitude>scores (OQ, 
Pilot, and Nav-Tech). academic major, arid institiitiotKii selectivity on program attrition rates arc shown in 
the Appendix (Table A3) and summarized m Table 4. Tliere was a- remarkable similarity of results within 
each of the programs. The full prediction nx^del containing all elepicnts of information available for each 
student yielded significant predictions of overall success in each program. In subsequent comparisons, the 
unique effects attributable to the Officer Quality composite and acaden^ic major were found to be 
significant in each of the prograns. In none of the samples were Pilot. N;fv-Tech scores, or institutional 
selectivity found to contribute independently to the prediction system. < 



/ , - Table 4. 


Summary 


of Regression Results 








Sl9nlflc«nca Lavals Within Samplas 




Four V«j 


ir Thr«« YMf 


Two Vaar 


Sourc* of Effact 


SchoMir«hlpt Scholarships 


Sc^holarshlps/Ot hor 


All Effects Combined. 








P\Jot and Nav/Tech Composites 


ns 


hs 


ns 


Institutional Selectivity (ACT) 


ns 


ns \ 


ns 


Academic Major {SSlE vs Other) 








Ofricer Qiiality Composite 








^Significant a( ihc .O.S Icvt-I. 








♦•Significant at the .01 level. 








"*No n -« gn i fi ca n t . 









The final equations identified as predictive of success in each^progrom are shown in Table 5 and 
plotted in Figures 1 through 3^L661cmg^nnfart^ scfiolarship recipients (Figure 1), it will be noted 

that the prbbability of successful completion was an ascending function of scores on the Officer Quality 
compofite for both S&B and non-S&E students. The specific function relating OQ to successful completion 
WIS .nearly linear for the'S&E participants ranging from an expected Value of .145 for persons scoring at the 
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Tabk ^, Final Regression Equations for Estimating Probability of Success in AFROTG* 



Aeactomle M«]or . 



^'pyy*'" 5cl«ne» »Wd Cnslntiring > Non-Seltnot"and«nglhMring 



Foiu Y^ ' r = U.39964 + .005 3467(OQ) - .0000527(OQ»> 7 = .154754 + .0130321(00) '- ,0009900(OQ') 
||uec Ycar. . 7 = .'428790 + .0105254(CXJ) - .0006354(Op») 7 = .776054 + .0020036(OCD - .0000882(OQ') 
fwoYeWOfher 7 = .664925 + .0075035(OQ) - .0005112(60^) 7 = .666661 + .0075663(OQ) - .0005597(OQ») 



Not*; ~ For applications where aca.demic major may not be known, less^accurate but nonetheless serviceable pre- ^ 
dictions may be^o^ained &om the following equations based on lifiodel 5: » ■ _ 

Four-ycar 7 * 161308 + .0105650(OQ) - .007041 0(dQ' ) ; 

Three-^ear 7 = .688214 + .0043434(OQ) - .P002525(OQ? ) , - 

. Two-year 7= -667319+ .00.74751(qQ) - .0005396(OQ») 

. ^Bas^d on model 3 described in Appendix A (Table 2). 

■" ^ ■ "■■ ' " , . ■ 

. 01 ^percentile level to .a high of .600 at the 95th percentile level. For students enrolled uhAOOrS&E curricula, 
: iht expected prdbability of completion increased from .168 at the 01 level to a maximum of .584 at the 
65tH percentile and decreased ^sUghtly thereafter. Once the threshold at 65 was , reached, no further 
improvements were noted in .the^robability of success among these students. 

■ ^■ 

In the 3-year program (Figu/e 2) differential probabilities of success were again, noted throughout the 
: range of OQ scores regardless of^kaidemic major. Differences between categories .of academic major were 
evident .to the extent that the prob^ilities of completion for S&E majors were consistently lower than for 
^iion-S&& majors throughout the entire rdnge of OQ scores. That is, at Tixed levels on the OQ composite, 
students enrolled in science and engineering cj^rricula were less likely to complete AfIIOTC than were 
students enrolled in non-technical areas. The probability of completion for S&E students increased from' 
approximately .44 to" .86 althou^ little further improvement was noted b^ond the 75th percentile. For 
non-S&E students, the proportion completing training increased, from .78 at the 01 level to approximately 
.89 at the 95th percentile. ' 

^Amon^ 2-year scholarship and other contract students (Figure 3), there was again an ascending 
relationship between OQ percentile score and probability of success in the program. Unlike the previous 
two samples, however, the S&E versus non-S&E distinction seemed to have little bearirtg on completion 
once the OQ level was fixed. For both groups, ^probability of completion increased from .67 to 
approximately .93 at the 65th percentile and evidenced very little improvement thereafter. The numerical 
values linking aptitude and academic major to the probability of success in each of the three programs are ^ 
summarized in Table 6. . . 

The relative efficiency of the selection system identified in these analyses, for discpminating 
successful versus nonsuccessful participants during the FY 71 through FY 75 time period is depicted in 
Tables 7 through 9. These tables show the frequencies, cumulative frequencies, and cumulative percent of 
, students scoring at each OQ level by academic major and twining outcome. Also shown are the actual and 
^•predicteci 'graduation rates by OQ level for S&E and non-S&E students. For example, the effect of a 
simulated requirenient that all S&E participants in the* 4-year program (Table 7) attain a score of 35 or 
better oh the OQ composite would have been, to eliminate 12.8% of the total S&E group (i.e., the 
cumulative percentage of all S&E students scoring 30 or below). ;At the same. time, the re^uirement.would 
have eliminated 16.8% of the eliminees versus only 7.9% of subsequent graduates. Expressed somewhat 
differently, it would also have had the effoct of eliminatyig all applicants whose predicted probability of 
> completion (expressied as a percent) was 29.5% or below. , » 

For non-S&E students, the same requirement would have identified 19.2% of the^elimin^s as 
opposed to 12.7% of the graduates. Similar interpretations can be made for the other percentile levels 
shown in the table. The effects of various simulated requirements on the 3-year program and the 2 -year 
program are shown in Tables 8 and 9, respectively. * 
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Joint Selectioii tot AFROTC and 
I^ted Training Prognuns 

It is common practice, particularly when dealing with scholarship students, to require that certain 
.candidate^ enter rated training programs (pilot or navigator) upon eventual entry to active duty. Thus, it 
would be important to consider the feasibility of selecting these students on the basis of their joint 
probabilities of completing both AFROTC and rated training programs. From previously unpublished 
analyses of ROTC graduates in the Air Force, it was found that success in' pilot and navigator training could 
be estimated from the Pilot and Navigator-Technical |pmposite scores, respectively, in much the^same 
fashion as was, done in the piesent analysi^'of AFI^TC completion rates. Table 10 presents the empirical 
findings of this research wherein the probability of completing (rated training is estimated from percentile 
scores on the appropriate composite. These estimates are "basfed on all AFROTC participants in rated 
training during FY 69 through FY 74 (N = 7,986 pilots and 1,924 navigator^), .Additional findings 
suggested that the prediction systems for both AFROTC and rated training were sufficiently independent 
of one another to permit the computation pf joint probabilities of cbmpfoHon as shown in Appendix B. 



fable 10. Probabilities of Successful Completion 
of Rated Training Program^ (UPT/UNT) as a Function 
of Percentile Scores oii the AFOQT Pilot 
and Nay-Tech Composites* 
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^Bosed on ROTC participants in rated training programs duing FY 69-FY 

PUot^uation: 'yp = .609595^.0026514 (Pilot); ^ ' . 

Navigator Equation: fN,= .714993.+ .0927026 (Nav-Tech) ' ' 



These tables show, the joint probability of completing both the 4-year scholarship program and rated 
training: Table Bl-for use with S&E students expected to enter undergraduate pilot Uaining (UPT); Tablfe 
B2-for use with non-S&E students scheduled to enter UPT; Table B3-for ute with S*E students scheduled 
;to enter undergraduate navigator training (UNT); Table B4-for use with non-S&E studeaits scheduled to 
enter UNT. • \ , 
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H^or example, in Table 'B I, the probability of completing both AFROTC arid UPT given OQ = 70 and 
. Plldt* 55 is shown to be .37. Similarly, the probability of completing both training programs given 0Q = 
; ■ ^ 40 and Pilot = 95 is .30. Similar tables for application with the 3- and 2-year programs could be constructed 
^ . ^ by simply crow-multiplying^ the appropriate cplumns in Tables 6 and 10. 

\^ Some discretion, Jiowever, should be exercised in the use of this information in an operational setting 

because of the implied value judgements associated with these tables. While two people may have the same 
; . probability of completion (e.g., 0Q= 55; Pilot = 25 vs. OQ =40; Riot = 75), it does not necWarily fo 

. * that the two candidates have equal yalue to the Air Force. It might be more desirable (and eventually less 

costly in terms of attrition) to adWt the candidate with 0Q = 40; Pilot = 75 in preference to the one with 
Pilot = 25 since relatively larger*attriti,on .costs ire^normally ass<fciated witti the pilot training 
" programs in comparison with the AFROTC scholarship program. , ' 

As a general rule for applying these data,-the operating agency must cohsidei;*the relative value to the 
Air Force associated with each of four joiot-t raining outcomes: 

, ^ (0, ) Passed AFROTC-passed pilot training 

(02) Parsed AFROTC-fafied pilot training 

(03) Failed AFROTC-would"have passed pilot training 

(04) Failed AFROTC-would have failed pilot training- 

Once specified, the values may then be* combined with the corresponding probabilities to yield the expected 
* . / value(EV)for a potentialtandidate: ^ 

■ ■ '" ■■ ■ ' . • ; ■■■ ■ ■ '/ ■ f " \ ' • ° ' 

■■^ ■ ' C/ -4.: .■■ r ■ 

EV= S V(0.)P(0.) . . / 

; -■■ ■ . ■ ' • " ; - 

where V(Oj) is the value of outcome Oj and P(a) iS'the possibility of outcome 0^. The P(Oj) for each of the 
four possible training outcomes is computed as follows: 

V . P(0, ) = Prob of passing AFROTC xProb of passing UPT/UNT 

P(02) = PrQfc of passing AFROTC x(l-Prob of passing UPT/UNT) * . 

PCOa) = (1/ Prob Qf passing AFROTC) X Prob of p^g UPT/UNT 
P(b4 )= (i - Prob of passing AFROTC) X (1 - Prob of passing UPT/U^ 

The probability for successful completion of both AFROTC and UPT/UNT, designated P(Oi), has been 
computed for the 4*year scholarship recipients in Tables Bl throiij^ B4 based on the independent estimates 
of success found in*Table 6X AFROTC) and Table 10 (UPT/UNT)/ The temaining probabilities designated 
V . P(02 ) through P(04) would be obtained by substitution in the. formulae shown above,^ 

*^ * To illustrate the approa^ consider acsituaripri where instead of maximizing expected value, program 

managers want to minimize the expected 3pst (EC) associated with each dedsion. The siame procedures 
would be followed exQept that," among a grven^set of applicants, the object would be to choose those 
representing the minimum' EC» Fuijher suppose that the average .outrof-pocket cost for each attrition in the 
^ , 4-year scholarship prggram has been estimated at $3,000 while cdhesponding costs for each attrition in 

' UPT might be on the order of $17,000. Assuming equal losses of $3,000 for outcomes O3 and O4 and zera 
> ^ . loss for outcome Oj , then the expected cost of selecting a given eandidate.would be: 



EC = s c(()pp(Oj) • ,^ ^ r 



See Grtfss and Su (1975) or Peterson (1975) for a more complete specification of procedures for yicorporatiijtp 
utility estimates into a general selection system. ^ 
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where the C(Oj) arc separate cost estimates for each outcome (0.): 0, 17,000, 3,000, and 3,000, 
lespectlVely. Taole/ 1 1 summarizes the costs and probability values for two hypothetical candidates. Both 
•le prospective science and engin^iing students designated for eventual entry into UPI. Candidate A 
obtains OQ = 55; EUot = 25 while candidate B obtains OQ = 40; Pilot = 75/ From Table 6, candidate A is 
found to haye a probability- of .418 for completing a 4-year scholarship while B has a probability of 
completion equal to ".345.^ .From' Table 10, the probability of completing UPT given Pilot composites of 25 
and 75, respectively, are .676 for A and .80^ for B. Computing expected costs for each candidate, it wili be 
noted tiiat since A has an EC = 4,120 as compared to 3,140 for B; pandidate A would be less desirable even 
Ifaougli bpth candidates had the same estimated probability of completing both AFROTC and UPT (P[Oi ] 
* .28). The* use of actual cost factors (or utility values) wouldj of course, yield different and perhaps more 
appropriate strategies foir join t^selection. / 

To^fe i.i. Dlastration of Expected Cost Compbtations .3* 





Candidate A: 


Candidate B: 






SolanM and EnglnMring Major 


■ Scianoa and Enginaarlng Malor 




cottf 


' . . wnnOQaSSi Pilot ■25 

3a ■ . . . » 


' ' wKh OQ ■ 40} Pilot ■ 75 

- J. 





C(0,>= 0 

C(Oa)= 17,000 
C(Oj)= 3,000 
C(04)= 3,000 



= •.28 



(.418) " (.676) 

P(02)= (.445) X (1 -.676) =.14 

P(03) = (1 - .418) X (.676) = .39 

P(04)= (1 -.418) X (1 -.676) =.19 



EC 



= .2 C(a)P(0.)=4l20 
J=l J J 



AFROTC 
(.345) 
(.345) . 
(1 - .345) 
(1 - .345) 



4 

EC = 2 
j=l 



■ UPT" 
(.808) 

(1 - .808) 
(.808) 

(1 - .808) 



:) = 3140 



= .28 

= .07 
= .53 
='.12 



Table 6. 



'Probabilities of completing AFROTC trainirtg at given percentile scores on the OQ composite are obuined from 



I'Pr obabilities of completing yPT.at g^ven percentile scores on the pilot cqmpdsite are' obtained froiii Table 10. 
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rv. SUMMARY AND CONCLUSIONS 

The principal conclusions reached as a result of these analyses may be summarized as follows: 

1. V^th prior knowledge of APOQt-OQ scores and academic major categorized as science and 
engineerihg versus x)ther curricula, it is possible to predict the probability of success in "AFROTC 
scholarship ptograms with a significant degree of accuracy. 

2. In general, there were found to be positive relationships between the AFOQT-OQ percentile score 
and training success in each of the programs included for analysis: 4-yjear schdarships, 3-year scholarships, 
and 2-year scholaiships/other contract studepts. 

v: . 3. Functional rektionaWps, between tl^ OQ composite and successful co^ 

•■"to vaiy by length of scholarship and by' academic major/ In generid, the effects of aptitu.de were more^r 
■ • pnmounced in the 4- and- '3-year pre^grams. Within ^scholarship programs, the relative likelihood df 
completing training at fixed aptitude levels was fomS to be lower for students enrolled in science and 
engineering couises than for those pursuing other academic majors. The evidence does not suggest whether 
this differential is the result of the relative ^difficulty of 'the two academic programs or whether S&E 
Qtatudcnts arc less motivated to complete training as compared to lion-S&E students. Further research on ^his 
i^e seems warranted. * . 

. • , . • 29 . 
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4. Wi^ one exception, the Effects of aptitude on the probability of completing training appeared to 
be asymptotic. Thsit is, the probability of success in most programs, increased with higher aptitude scores up 
to a certain point, after which, Uiere was^^jittle Qi^'no inaease in the expectancy for completion. Here again, 
alternative explanations for thi^se effects are plWibe. The aptitude requiremeAtf bP the various academic 
programs may be such that additional talent beyond a fixed level may not materially affect the likelihood 
of success. On the other hand, students with higher aptitudes may be less motivated to complete the 
AFROTC program. Either one or both of these conditions may be operative. Among 4^ear scholarship 
students pursuing science and. engineering degrees, the effects of OQ on probability oroon;ipletion were 
more nearly linear throughout the entire range of aptitude scores. , / . ■ % i ' * 

( ;v ;$. Once the effects of the OQ composite and academic major were , accounted for, no significant 

.: increase in pjedictiye accuracy was obtained using ieith^r the Pilot' or . Navigator composite or the index of 

institutional selectivity. The^ frndings in^icatedfthat^eiiid^dns b^d on OQ and academic m^or would 

be appUcabJe/regardl<^ss of ^or^s obt^ed on the flylog training composites and regardless of the degree of ^ 

selectivity. ebceicised by rite host liristitu^ 

• .' ■ ' • * ' -.■ ■ . 

6. Although not 'essential for predictjiig training outcomes in AFROTC, the AFOQT Hlot and 

Navigator compositesorMf^ shown to be effective in, estimating whether or not an applicant will eventually , . 

complete undergraduate pilot and navigator training, tfnplications of the results for establishing multiple 

criteria for joint selection into AFROTC and subsequent flying training programs were discussed within a 

general utility framework. 

Based on. these analj^s, it is recommended that the AFw^T coinp<>s&te8 and supp^^^ 
information on intended ac^^c major be included in the seljectionWsteni fpr^scholitfship awards. Such 
actions would a^st in ident^ying "Wgh risk^' caindi'aates prior to actual award of scholarship benefits. 
Further i^fforts to refine thf^^^lection system based ori. analyses of add^onal predictor variables appear to 
warraiftejd. , ^ 

. > 
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APPENDIX A: TECHNICAL SPECIFICATIONS FOR REGRESSION ANALYSES 



The tables and illustration shown^ Appendix A dfilcribe the technical aspects of the regression 
analysis. Table Al lists predictor variables. Table A2 describes the prediction models that were generatedi 
and Figure Al shows the sequence of statistical comparisons between models. The five hypotheses tesjted 
witWrt each <^f t^e4;,4-f arid 27earschoU^ . \ - ;. ' . 

\. Test for Overall Effects - In this c9mparison, the effects of all predictor variables coihbined were 
tested for statistical sigmficancc. Negative findings in this cornjfarison would have precluded further testing 
. within the sample. . . • , . : 

2. Test for AFOQT Pilot and Nav-Tech Effects '- In this comparison, the effects of the .AFOQT 
flying training cbmposites were tested holding effects of all other variables constant. Negative 'findings 
would have indicated that information on the Pilot and Nav-Tech compc tc^ provided no unique 
contributions to predictive accuracy in the context 6f the Amaining variables. ^ . 

" ; 3 . ' Test ^ for Effects Attributable, to Institutional Selectivity (ACT) - This comparison was designed to 
test foFUiaique effects assoiciatcdjyvith differences in the input quality between institutions. Negative 
Vmdings woiud^i;^ implied that functional relationships between the remaining predictors and training 
outcome were siniMll^for alllevels of institutional selectivity. ^ 

4. Test fox Effects due to Academic Major (ME vs. other) - In this comparison, thlTunique effects 
w associated with acadernic major wert tested at fixpd levels on the remaining variables. 

■' 5. 7VV for AFOQT-OQ Effects - This comparison was designed to' .'test tor unique effects of the 
Officer Quality composite in determining final training outcomes. 

As can be noted in Figure- Al, the specific modeb used to test each successive hjf^thesis were 
predicated on results from preceding comparisons. The dotted line represents the actual sequence of 
comparisons based on results within'each of the three scholarahipsamples..That is, Model 3 >yas foOnd to be 
the rntfst ^propriate for predic tion pur poises within each sample. Additional details of the statistical 
* pnjccdure ma/be foundin Ward and Jerirungs (1973).- ... S*' 

Table AL Predictor Variablds ^ . 



Varlabift V Ottcrlptlon 

1 AFOQT-OQ * 

2 AFOQT-OQ Squared 

3 AFOQT -PUot 

4 ' AFOQt - Pilot Squared ... 

5 ' AFOQT - Pilot Availability (1 if score available; 0 otherwise) 

6 AFOQT.Nav/Tech ° v . - , , ; • 

7 AFOQT-Nav/Tejch Squared 

8 AFOQt-Nar/Tech - Availability ( 1 if score available ; 0 otherwise) 

9. Average ACT Composite — varies by institution ' • 

10 v/. Average ACT Composite Squared 

11 ^ienca and Engineering Academic Major ( 1 if S&E; 0 otherwise) ^ 

1 2 Non-Science and Engineering Academic Major (1 if non-S&E; 0 othSrwise) 
13^16 OQby ACT (Var 1-2 xVar 9-10) 

1 7-20 OQ by Academic Major (Var 1 -2 x Var 11 -1 2) 

21-24 • ACTby Academic Major (Var 9-1 Ox Var 11-12) 

25-32^ - OQ by ACT by Academic Major (Var 9-10 x Var 1 1-12). 

33-36 Pilot by ACT (Var 3«^ x Var 9-10) 

37-40 PUot by Academic Major (Var 3--4 x Var 9-10 x Var 11-12) 

41--48 PUot by ACT by Academic Major (Var 3-4 x Var 9-10 x Var 1 1-12) 

49-52 Navby ACT (Var 6-7 X Var 9-10) 

53-56 Nav by Academic Major (Var 6-7 x Var 1 1 -12) 

57_64 Nav by 'act by Academic Major (Var 6-7 x Var 9-10 x Var 11-12) 

65^ . Training Outcortie (1 if graduated ; 0 otherwise) 

— • ^ ^ " 5t 



^igm Al. Flow char for model comparisons. 



Table A2, Regression Models 
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OQ ' • . . .. 
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•PLT + NAV • t. ,. 
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OQ + ACT + PLT + NAY + (All interactions) 
OQ + PLT + NAV:v . . , ' ■ * * , ... 
MAI +-PLT + NAV + (All interactions) ' ' . .' 
ACT + PIT + NAV + (All inteiractions) 
MAJ + ACT + PLT + NAV + (laiijiteractions) 
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l-if^urc H). Flow chart for model comparisons. 
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APPENDIX B: JOINT PROB ABIUTY t»fcBSf OR ESTIMATING 
COMPLETION OF BOTH AFROTC (FOUR- YEAR SCHOLARSHIP PROGRAM) 
AND UPT/UNT BASED ON AFOQT-COMPOSITE SCORES 




Tabie BL Probablitks for Sucmsful CompitHon of Both the Four-Ycar Scholarehip 
Program and UPT as a Function of Percentile Scoiti on the AFOQT-OQ 
and Pilot Comppritcs - Science and Engineering Mijorp 
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P|: Marginal probability of completing AFROTC at given OQ percentile scores. 
Pj: Mn^inal pr«^bability of completing UPT at given pilot percentile scores. 

,. I, ■ ' ■ ,. . 

Tabic Bl Probabilities for Successful Completion of Both the Four-Year Scholanhip 
Pro9^m,and UPT as a Function of Percentfle Scores on the AFOQT-OQ and POot 
Composites-Non-Science and Engineering ftfajors 
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Pi: Marginal probability of completing AFROTC at given OQ percentile scores. 
Pjc^Marginal probability of completing UPT at given pilot percentile scores. 
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P, : Marginal probabUity of completing AFROTC at given 6q pcrcentUe scores. 
P,: Marginal Piobability of completing UNT at given Nav-Tech pcrcentae. 

Table B4. Probabilities for Successful Completion of Both the Four Year Scholanhip 
Program and UNT as a Function of Percentile Scores on the AFOQT-OQ and Nav-Tech 
. Composites (Non-Science and Engineering Majors) 



APOQT Nav« Tach C^mpotlta 



AFOQT*OQ 
C«m|Mfllt» 



.<20 2% 30 3S 



40 


4S 


SO 


ss 


SO 


SS 


70 


7S 


SO 


ss\ 


.14 


.14 


.14 


,15 


.15 


.15 


.15 


.15 . 


.16 


.16 


.18 


.18 


.19 


.19 


.19 


.19 


.20 


.20 


.20 


.21 


.23 


.23 


.23 


.24 


.24 


.24 


.25 


.25 


.26 


.26 


.27 


.27 


.28 


.28 


.29 


.29- 


.30 


.30 


.31 


.31 


.31 


.31 


.32 


.32 


.33 


.33 


.34 


.35 


.35 


.36, 


.34 


.35 


.36 


.36 


.37 


.37 


.38 


.38 


.39 


.40 


.38 


.38 


.39 , 


.3»^ 


.40 


.41 


.41 


.42 


.43 


.43 


.40 


.41 


.42 


.42 


.43 


.44 


.44 


.45 


.46 


.46 


.43 


.43 


.44 


.45 


.45 


.46 


.47 


.48 


.48 


.49 


.45 


.45 


.46 


.47 


.47 


.48 


.49 


.50 


.50 


.51 


.46 


.47 


.48 


.48 


.49 


.50 


.51 


.51 


.52 


.53 


.47 


.48 


.49 


.49 


.50 


.51 


.52 


.53 


.53 


.54 


.48 


.49 • 


.49 


.50 


.5 1 


.52 


. .5-2 


.53 


.54 


.55 


.48 


.49 


.50 


.50' 


.51 


.52 


.53 


•.54 


.54 


.55 


.48 


.49 


.49 


.50 


.51 


.52 


'.53 


.53 


.54 


.55 


.47 


.48 


.49. 


.50, 


.50 


.51' 


.52 


.53 


.54 


.54 


.46 


.47^ 


.48 


.49 


.49 


.50 


.51 


.52 


.53 


.53 


.45 


.46 


.47 


.47 


.48 


.49 


.49. 


.50 


M 


:* 


.43 


v44 


.45 


.45 


.46 


.47 


.47 


.48 


.49 


.47 


.41 


.42 


.42 


.43 


.44 


.44, 


.45 


.46 


.46 



SO. ss 



01 
05 
10 
15 

• 20 
25 
30 
35 
40 
45 
50 
55 
60 
65 

• IXL 
75 
80 
85 
90 
95 



.13 
.17 
.21 
.25 
.29 



38 
40 
42 
43 
44 
45 
45 
45 
44 
43 
.42 
0 
38 



.13 
.17 
.22 
.26 
.29 



.32 .33 
.35 .36 



.38 
.41 
.42 
.44 
.45 
.45 
.46 
.45 
.45 
.44 
.43 
.41 
.39 



.13 

.17 

.22 

.26 

.30 

.33 

.36 

.39 

.41 

.43 

.44 

.46^ 

.46 

.46 

.46 

.46 

,45 

.44 

.42 

.40 



.14 

.18^ 

.22 

.27 

.30 

.34 

.37 

.40 

.42 

.44 

.45 

.46 

.47 

.47 

.47 

.47 

.43 
.40 



.40 
.44* 

.47. 

.50 

.52 

.54 

.55 

.56 

.56 



.18 
M 
.27 
.32 
.37 
.41 
.44 
.48 



.-^.769 .782 . 793 .810 .823 .837 .850 .864 .877 .891 .904 .918 .931 



.55 
.54 
.52 
.50 
.48 

945 .958 .972 



.54 
.56 
.56 
*^57 

M 

.56 
.55 
.53 
.51 
.49 



.168 
,218 
.275 
.328 
.376 
.449 
.457 
,490 



.50 .518 
.53 .541 



.559 
\5n 
.580 
.584 
.581 
.575 
.564 
.547 



P^: Marginal probability of 
Pj : Marginal probability of 



completing AFROTC at given OQ percentile scores. , 
completing UNT at given Nav»Tech percentile scores. 



